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ABSTRACT 

A large-scale suaaative, coaparative evaluation of 
"Patterns in Arithaetic" (PIA), a aodern televised arithaetic 
curriculutt for grades 1-6 was carried out in grades 1-4 in both rural 
and urban schools during the 1970-71 school year. This report deals 
with the results of aonitoring procedures carried out, and student 
achieveaent as aeasured by pre- and posttesting using standardized 
instruments. lapleaentation data were gathered by interviews with 
teachers and principals, teacher questionnaire, exaaination of 
records, and direct observation. In four of the five sites and 
lapleaentation requisites were judged to have been satisfactorily 
fulfilled. Achieveaent data were gathered on three coaponents 
(concepts, coaputation, problem solving) and analyzed using 
■ultivariate analysis of variance. Results indicated that the 
achievement of students in the control group was about equal to that 
of students in the experimental group. The results did not vary by 
sex, grade level, or site. (Author/SD) 
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STATEMENT OF FOCUS 



Indlvidiiaily Guided Education (IGE) is a new comprehensive 
system of elementary education* The following components of the 
IGE system are in varying stages of development and implementation: 
a new organization for instruction and related administrative 
arrangements; a model of instructional programing for the indi*- 
vidual student; and civrriculum c^Tiponents in pn^ readings reading^ 
mathematics, motivation, and environmental educvtion. The develop- 
ment of other curriculum component of a system for managing in- 
struction by computer, and of :instructional strategies is needed 
to complete the system. Continuing programmatic research is required 
to provide a sound knowledge base for the components under develop- 
ment and for improved second generation components* Finally^ sys- 
tematic implementation is essential so that the products will function 
properly in the IGZ schools* 

The Center p^ans and carries oi/t the research, development, 
and implementatiotx components of its IGE program in this sequence: 

(1) identify the needs and delimit the component problem area; 

(2) assess the possible constraints— financial resources and avail- 
ability of staff; (3) formulate general plans and specific procedures 
for solving the problems; (4) secure and allocate human and n^terial 
resources to carry out the plans; (5) provide for effective coramuTii- 
catlon among personnel and efficient mi^nagement of activities and 
resources; and (6) evaluate the effectiveness of each activity and 
its contribution to the total program and correct any difficulties 
through feedback mechanisms and appropriate management techniques. 

A self-renewing system of elementary education is projected in 
each participating elementary school, i,e*, one which is less dependent 
on external sources for direction and ia more responsive to the needs 
of the children attending each particular school* In the IGE schools. 
Center-developed and other curriculum pi^oducts compatible with the 
Center's instructional prograiaing model will lead to higher morale 
and job satisfaction among educational ^^ersonnel* Each developmental 
product makes Its unique contribution IGE as it is Implemented In 
the schools. The various research components add to the knowledge of 
Center practitioners, developers, £.nd theorists. 
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ABSTRACT 



A large-scale suramative, comparative evaluation of Patterns in 
Arithmetic (PIA), a modern televised arithmetic curriculum for Grades 1-6, 
was carried out in Grades 1-4 in 92 rural and urban schools in five sites 
during the 1970-71 school year. About 5,000 pupils in the experimental 
schools received PIA as their basic mathematics course while another 5,000 
students in control schools continued to use conventional arithmetic pro- 
grams. Achievement wa£ measured by standardized tests of mathematics 
concepts, computation, and applications in problem-solving. Results indi- 
cated that achievement of control school pupils was about equal to that of 
students in the experimental group. The results did not vary by sex, grade 
level, or site. 
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INTRODUCTION 
Background 

Patterns in Arithmetic (PIA)^ a comprehensive elementary mathematics 
program for Grades 1-6, was originated in 1959 and developed over the next 
decade under the leadership of Professor Henry Van Engen with the support 
of the Ford Foundation and, starting in 1964 ^ the Wisconsin Research and 
Development Center for Cognitive Learning. The components of PIA are 
video-taped lessons which are usually presented via broadcast television, 
teacher suggestion manuals, and pupil exercise books. Unlike many curric- 
ula which employ television, PIA is designed to be a complete*, program of 
elementary matheanatlcs rather than a supplementary or enrichment series* 
When PIA is used as a basic Instructional program, a class watches a 
IS^minute telecast once or twice a week (the schedule varies at different 
grade leveliO and the teacher then develops the lesson content by using 
the student exercise books and activities from the manual. The normal 
sequence of events would be <1) a few minutes of pre-telecast orientation, 
as suggested in the teacher's manual; (2) the telecast, during which the 
TV teacher may ask the pupils to manipulate various objects or work a few 
problems; (3) post-telecast activities, also suggested in the manual, which 
enlarge upon or review the important concepts presented in the telecast; 
and (A) over the next few days, supplementary activities from the manual 
and exercises in the pupil exercisa books, some of which are related to 
material In the telecast and some of which are review of concepts presented 
In previous telecasts. Short tests are also given a few times a semester 
so that teachers can assess pupil progress* 
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PIA was designed to be utilized in the period of transition from a 

traditional mathematics program to a modern mathematics program at the 

elementary level* To quote Professor Van £ngen> 

Remember that the telecasts are not intended to replace 
the classroom teacher « They are intended to help intro- 
duce and demonstrate new mathcoQatical ideas. You, the 
teacher^ are still the most important element in the 
development of a sound and meaningful mathematics pro- 
gram in your school* »PIA is one answer to a new pro- 
gram in mathematics^ but after having used PIA for a 
year or two, you will be ready to find other answers for 
your school. Whatever happens^ remember that change is 
inevitable* You must help determine the direction of 
that change [1967, p* vi]* 

The dual aims of PIA are to provide a sound program in mathematics 
for the elementary school child and to provide inservice education for 
the elementary school teacher* Television was chosen as the meditan by 
which to accomplish these aims; it was noted that televised programs are 
particularly effective for pupils and teachers in rural areas and in the 
centers of large cities. The attribute of a rural setting that makes 
televised education promising is that without it, rural teachers may not 
have opportunities to update their skills and, consequently, students may 
not receive modern mathematics instruction* Used properly^ — ^with the 
teachers viewing the program as their pupils see lt~PlA thus becomes an 
Inservice program in the concepts and pedagogy of modern mathematics* 
Both teachers and pupils learn the new ideas through the medium of tele- 
vision from qualified teachers presenting contemporary mathematics content* 

Urban settings are often characterized by high transiency of pupils, 
and even teachers, within the school district. A televised program, in 
addition to being a familiar and appealing medium, provides for continuity 
in the basic skills program and is an educational touchstone for the tran- 
sient pupils* Moreover, since urban children are frequently deficient in 

ERLC 
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the reading skills required for efficient learning from mathematics text- 
books » a program such as PIA which features video-taped lessons, activi- 
ties with stress on the manipulation of concrete materials, and exercise 
books with minimal reading may be a particularly effective means of pre- 
senting new concepts for these children. 

There are some factors, of course, which are Important in both types 
of demographic situations — television's appeal to children and its effi- 
ciency and economy as an inservice course. 

For more complete information on the history, msithematical content, 
behavioral objectives, and pedagogical principles of PIA, see Braswell and 
Romberg (1969). For an overview of the key mathematical ideas in PIA, see 
Appendix A. 

Previous Evaluations 

As part of the research and development effort of PIA, a summatlve, 
non-comparative evaluation of the program in Grades 1 and 3 was carried out 
during the 1966-67 school year (Braswell & Romberg, 1969). Several hundred 
first- and third-grade classrooms in communities of four different sizes in 
Alabama and Wisconsin took part in the study; the two largest communities 
were broken down into three socioeconomic levels. The results of that 
study indicated that most of the PIA program objectives were being accom- 
plished. In addition, further analysis revealed no relationship between 
performance and community size or socioeconomic level. The only difference 
appeared to be between the two states, and then only on the standardized 
computation test . 

A field test for Grade 2 was carried out In the 1967-68 school year 
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<Braswell» 1969b)* Again, the results indicated that PIA was acco^rplishing 
most of its goals » At the end of the school year^ the children in the 30 
classes in the study scored significantly higher than the test aonns on a 
standardized test of concepts for that grade level, even though they had 
scored lower than the test norms at the beginning of the year* 

A formative evaluation of the PIA program for Grade 5 was also carried 
out in 1967-68 (Braswell^ 1969a) j about 80 classes participated* Results 
indicated that children learned traditional computation skills and impor- 
tant concepts in arithmetic coimnensurate with their stage of development. 
Computation problems tended to be easier at the end of the school y<^ar than 
they vere shortly after the skills were presented, which is good evidence 
that skills learned early in the year are not forgotten but are reinforced 
by the structure of the PIA pupil exercise books. 

In the 1968-69 school year, the PIA staff at the Wisconsin Research 
and Development Center for Cognitive Learning conducted a formative evalu- 
ation of the PIA program for Grade 6 (Braswell, 1970). The study success- 
fully used the techniques of item sampling to evaluate the year's curriculum 
and to engender several specific improvements in the telecasts^ 

Reasons for the Present Study 

One of the original aims in developing PIA was to provide inservice 
education for elementary school teachers, but none of the evaluations men- 
tioned heretofore had inveBtigated the effectiveness o^ PIA in this respect. 
In addition, partially as a result of questions from prospective consumers 
regarding utilization of PIA in various settings, a need to ascertain the 
effectiveness of PIA in rural and urban sites, especially among children of 



lower socioeconomic background and achievement, was recognized. None of 
the previous studies had focused on this particular question. Moreover, 
there had never been an evaluation of the Grade 4 program and the Grade 1-3 
materials had not been evaluated following a 1970 revision. Thus, a large- 
scale summative, comparative evaluation, focusing both on the inservice 
effect on teachers and on the achievement of rural and urban pupils in 
Grades 1-4, was proposed and funded for the 1970-71 school year. 

Goals of the Study 

Plans for the study call for the data to be evaluated from three 
approaches: the effects of PIA on pupils, the effects of PIA on teachers, 
and the interaction between the two. This report will concern itself only 
with the first question? in what ways, if any, is student achievement 
affected by PIA? The second question, regarding the effects of PIA on 
teachers, is discussed elsewhere (Marshall & Fischbach, 1972); in summary, 
the results of that study indicated that with respect to PIA-specific 
content, but not to general basic mathematics, PIA is clearly an effective 
inservice course for teachers, particularly for teachers with relatively 
low initial knowledge of the basic mathematics underlying a contemporary 
program. Discussion of the third question, dealing with interaction be- 
tween teacher and pupil effects, will appear in a third report if the data 
are of sufficient interest. 

The specific questions to be dealt with in this report are: 

1. Do students who xxae PIA increase their achievement on standard- 
ized measures in any one or more of the following areas of 
mathematics? 

a. computation 

b. concepts 

c. applications /problem-solving 
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2. Bo students who use PIA i.^crease their achievement on program- 
related objectives In any owe or more of the following ar&as of 
mathematics? 

a% computation 
b. concepts 

c • applica tions/pr oblem^solving 

3^ Do the eff ects^ If any, of PIA on pupil achievement vary with 
any one or more of the following? 

a. demographic characteristics 

b. grade level 

c. sex 

Also included in this report is descriptive information concerning the 
field test sites and the mathematics program implementation in each. This 
information was gathered to secure a more comprehensive description of the 
participating schools and to verify that the PIA program was carried out 
according to the recommendations of the developers. It was also used as a 
basis on which to exclude data from the final analysis and to develop 
teacher implementation and program appropriateness scores which were cor- 
related with achievement in the schools using PIA. 
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THE EVALUATION PLAN 



Subjects 

The evaluation was designed to be carried out in urban and rural sites 
in which the teachers and/or students reflected in varying degrees and 
combinations the characteristics (e*g.» student lack of reading skills) dis- 
cussed in Chapter I« Site selection was somewhat restricted by the avails- 
ability of broadcast facilities and, of course » by the unwillingness of some 
school systems to commit large numbers of schools to an extensive one-year 
field test* However, enough distinctive sites were secured to enable 
potential consumers to generalize the results of the study to a variety of 
local situations. A Memorandum of Agreement between the Center and each 
site was prepared specifying their respective contributions to the field 
test (see Appendix B)« 

Urban sites selected were the Catholic Archdiocese of New York City 
Schools, the Chicago Public Schools, and the Portland (Oregon) Public 
Schools « Rural sites were located in the state of Vermont and in three 
rural counties near Roanoke, Virginia. In all sites except Vermont, most 
or all participating schools had qualified for Title I funds, indicating 
that a substantial number of the students were from impoverished homes and 
were below grade level in achievement. In four of the five sites, schools 
willing to participate were identified and were randomly assigned to the 
experimental (PIA) group or the control (non-PIA) group. In the fifth site 
(referred to later in this paper as urban site 9.) randdmlzation was not 
possible due to technical constraints caused by the geographic location of 
the schools with broadcasting facilities; thus, selection of schools for 
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the control group was of necessity dor.e through the cooperation of the 
central office personnel by matching school district with the experi- 
mental group in parental socioeconomic status ^ standardized test scores, 
etc. 

Approximately 10,600 children and 390 teachers in y2 schools took 
part in the study* Because of unforeseen causes this sample was reduced 
somewhat in the data analysis and one site (urban site 3) was dropped from 
the study entirely» but the nunibers in the analysis are still of the same 
order of magnitude. Table 1 indicii.tes by grade level the approximate num- 
bers of children, teachers, and schools in each site* Within each site there 
were approximately equal nuinbers cf experimental and control schools . In 
those sites where more than one grade level participated > there were approx- 
imately equal numbers of classes in each grade. It is important to point 
out that all four grade levels are not represented ir. each site; however, 
each grade level appe^irs in at least one urban and one rural site. 

Method 

Classes in the experimental schools used the VIA progr^ns as the sole 
or major component in the mathematics curriculum* This included utiliza- 
tion of the television programs, pupil exercise books, and the teacher's 
manuals according to the developers' recommendations incorporated in the 
requisites for Implementation, to be described later in this chapter* 
Classrooms in the control schools proceeded as usual with whatever mathe- 
matics curriculum they chose to use. Although teachers in control school 
classrooms were encouraged not to use the PIA telecasts ^ they were not 
prevented from doing so if they chose; in fact, some of these teachers did 
use the FIA telecasts as a supplement to their basic curriculum. 
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TABLE 1 

APPROXIMATE NUMBER OF PARTICIPANTS IN THE STUDY 



Grade Level 



* Site 




1 


2 


3 


• 

A 


Schools 


Teachers 


Pupils 


Urban site 


1 






X 


X 


46 


150 


4,700 


Urban site 


2 


X 


X 






9 


90 


2,350 


Urban site 


3* 


X 


X 






9 


40 


850 


Rural site 


1 


X 


X 


X 




10 


70 


1,650 


Rural site 


2 








X 


18 


40 


1,050 






TOTAL 






92 


390 


10,600 



^ata were not used in analyses* 

At the beginning of the 1970-71 school year^ informational meetings 
were conducted by Center staff for both experimental and control school 
teachers in all sites to explain the field test. The meetings for the PIA 
teachers also provided Inservice training in the utilization of the PIA 
program as specified in the teacher's manuals and elsewhere by the devel- 
opers (Braswell & Romberg, 1969). Schedules for student and teacher data 
collection for the year were outlined, and pretest /questionnaire data were 
gathered from the participants fpr the teacher effect aspect of the field 
test (Marshall & Fischbach, 1972). In spring of 1970 a final brief meeting 
was held at all sites, primarily to gather pofitvest teacher dati\ and review 
student post testing procedures, but also to report some preliminary results, 
particularly monitoring information, from the field test. Kepresentative 
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agendas and materials for both the initial and final meetings with 
teachers appear in Appendix €• 

During the year the Center provided no further Inservice training 
and local school central office personnel offered no more consultant sup- 
port either to PIA or to control schools than was typically given to other 
schools or during other academic years • However, Center staff members or 
field workers trained by the Center carried out monitoring visits in all 
sites to gather descriptive information and to ascertain whether the PIA 
program was implemented In the experimental groups according to recom- 
mendations detailed at the inservice meetings and in the teacher ^s manuals « 

Requisites for Implementation of PIA 

The recommended instructional procedures for PIA teachers as well as 
certain basic specifications for the field test were formalized in a state- 
ment of requisites for effective implementation* Judgments about satis- 
factory implementation and subsequent exclusion of data from certain sites 
or schools from the analysis ^ if necessary ^ could then be made on the basis 
of the requisites. Also^ teacher implementation and program appropriate- 
ness scores to correlate with student achievement could be developed from 
the requisites* The requisites were as follows: 

1* All PIA teachers should receive inservice training In PIA 
program utxllaation* 

2. All or most of the televised lessons and exercise book 
activities should be completed by all or most students at 
each grade level » 

3. Technical aspects of the PIA telecast t>resentatlon should 
be adequately fulfilled, 

a. Television reception should be consistently good. 

b. Telecasts should be broadcast according to the 
established schedule. 



c. Repeat telecasts should be available to allow 
alternative viewing times as well as to provide 
for absentees and for students who need the rein- 
forcement of a second presentation* 

The recommended PIA instructional procedures should be followed ^ 
including: 

a* A substantial portion of the background segments of 
the teacher^s manual (e^gi^^ the behavioral objectives) 
should be read by the teacher prior to the telecast « 

b. Telecast lessons should be preceded by an orientation 
or pre-telecast activity directly related to the 
objectives of the lesson. 

c. Telecast lessons should be followed immediately by a 
concluding post- telecast activity directly related to 
the objectives of the lesson. 

d. At least one "Highly Recommended" activity specified 
in the manual should be used in conjunction with each 
lesson for all or most students* 

e. At least some of the exercise pages associated with 
each lesson should be completed by all or most students. 

f « Common mathematics manipulatives should be used in 
conjunction with the lessons when specified. 

The ?XA program should be deemed appropriate for students of 
varying ability levels throughout the school year as evidenced 
by student attitude and by teacher judgment, as follows: 

a. Students should exhibit a positive attitude towards the 
telecasts. 

b« The pace of the telecast presentation should maintain 
student attention. 

c. The level of difficulty of student exercises should be 
appropriate* 

d. !Tse of supplementary printed materials from other curricula 
should be largely unnecessary. 

e. The level of difficulty of the objectives of each lesson 
should be appropriate. 

f . The time Intervals between telecasts should be adequate. 

A minimum of one-half hour of instructional time per day should be 
devoted by each teacher to the PIA program for all children. 
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Data Collection Schedule and Instrumentation 

All students in both VIA and control schools were given pretests of 
arithmetic achiev^nent in the fall of 1970 and posttests in the spring of 
1971 according to the schedule shown in Table 2* With the exception of the 
pretest for Grade 1, which was developed at the Center^ all of these instru- 
ments are well-known standardized tests: the California Achievement Tests— 
Hathematics (CAT)» Levels 1 and 2, Forn^ A and B> 1970 edition, published 
by the California Test Bureau; the Comprehensive Tests of Basic Skills — 
Arithmetic (CTBS), Level 1, Forms Q and R, published in 1968 by the Califor- 
nia Test Bureau; and the Cooperative Primary Test (Mathematics), Forms 12a, 
23a, and 23b, published in 1965 by Educational Testing Service (ETS)* The 
tests were selected with three considerations in mind^ First, they reflect 
PIA objectives to a greater extent than other instrximents examined. In 
particular, the Cooperative Primary Test and the PIA program both place 
emphasis on concept development*" -^^econd, they are quite widely accepted 
measures and thus, potential users of PIA would have confidence in them* 
Third, particularly in the case of the tests chosen for Grades 1-3, they 
require a minimum reliance on reading skills. Although more testing time 
was required, two tests were utilized in Grades 1-3 in order to adequately 
cover the range of PIA contents Analysis of the data was performed both 
for three subtests of the CAT and CTBS tests in their entirety— computation, 
concepts, and applications /probiem-solving^^-^and on several subscales 
developed for the PIA content valid it^s from the computation and concept 
sub tests « (See Appendix P f or a summary of the subscales formed from PIA 
content valid items*) It should be pointed out that the concepts and 
application/problem-Eolving sections of the CAT are usually treated as one 
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subtc^st since the national norms were Dstabliahed on this basis « However, 
since the present analysiB was performed on taw scores^ separate subtests 
could be formed for these two areas « 

The sources of descriptive and implementation data were interviews with 
principals and, teachers, observations of teachers, and teacher questionnaires • 
Both the interviews and observations were held on a continuing basis through- 
out the school year, while the questionnaires were administered at the inl^ 
tlal and end-of-»year meetings of participants at each site. One interview 
was held with the principal from each experimental and control school and 
with each experimental (PIA) teacher; about one-half of the control group 
teachers were sampled for interviews « A random sample of teachers drawn from 
the Interviewees In both groups were observed during their mathematics class 
instruction. Slightly less than one-half of the PIA teachers and one- third 
of the control group teachers were observed % All field test monitoring, sum- 
marized in Table 3, was carried out by Center staff or by field workers 
trained by the Center rather than by local school system personnel. Copies 
of monitoring Instrt^ents appear In Appendix £. 

It will be noted that certain questions on the monitoring Instruments 
deal with teacher and student attitudes toward PIA^ Attitude data for both 
PIA teachers and students were collected primarily to reveal any trouble spots 
in the Implementation program rather than to do a rigorous attltudlnal study, 
since previous field tests had already provided substantial evidence that PIA 
was positively received by both teachers and students (Braswell & Romberg, 
1969). Because In some sites teachers who had previously selected the math 
series for their students Independently did not participate in the decision 
to use PIA, it was felt that possible negative attitudes affecting teacher 
cooperation and implementation might be evident and should be Identified. 
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RBSUtTS OF HOMIT0RIK6 PROCEDURES 

Purpose of Monitoring 

Monitoring of the field tfcst was carried out to determine whether 
experimental (PIA) schools In each site effectively met the requisites for 
satisfactory program implementation and to secure further descriptive 
information about all participating schools, both experimental and control. 
The purposes and results of monitoring must be vi^ed In the context of the 
general design of the evaluation, particularly with regard to the requi- 
cites for PtA schools. Theoretically, we would expect that for both groups 
an equal possibility would exist for varying degrees of effective mathe- 
matics program implementation and that, consequently, student achieveoent 
data from no schools or sites could legitimately be dropped from analysis on 
the basis of Ineffective Implementation; thus, we would also expect that 
monitoring to assess implenentation practices would be essentially unneces- 
sary. However, because of a variety of factors^ — PIA was completely new to 
the teachers and a somewhat unusual program in that it utilized telecast 
instruction; teachers as a rule did not actively participate in the deci- 
sion to use PIA and in some instances used it against their will; and pos- 
sible technical difficulties were associated with the adoption of PIA-- it 
was believed not inconsistent with the evaluation plan to set some basic 
requisites for the PIA groups which, if satisfied, would guarantee a "fair 
trial" of the program and justify inclusion of the data from these sites in 
the analysis. An arbitrary decision was made i^ot to drop individual schools 
from the study on the basis of requisite information, but in the event of 
extreme cases of poor implementation Involving an entire site to exclude the 
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site entirely* Eventually all data from urban site 3 were omitted from the 
analysis according to this plan* The requisite information, summarized in 
Table A, and the implementation and program appropriateness ratings derived 
from it are reported here primarily for descriptive purposes « In Chapter IV 
these ratings are related to pupil achievement in PIA schools « 

Again, keeping in mind the overall plan of the evaluation and the ran- 
dom assignment to groups ^ it perhaps would not appear necessary to further 
describe the PIA and control groups via information from monitoring proce- 
dures. Note, however, as reported earlier, assignment in urban site 2 could 
not be random due to technical constraints, and therefore, the comparative 
data become more relevant. Also, where differences exist between experi- 
mental or control groups, whether due to chance or not, it is desirable to 
document th^* The comparative information summarized in Table 6 may also 
be of interest to prospective consumers of PIA who identify parallel condi- 
tions in their local situations. 

Implementation Requisites 

Table 4 summarizes the results by site for the experimental group for 
each of the six Implementation requisites detailed in Chapter II • The first 
requisite regarding inservlce attendance was simply a matter of record, and 
the second requisite (use of telecasts and workbooks) was examined by means 
of a questionnaire. Hie primary source of information from PIA schools for 
the remaining four Implementation requisites was the personal Interviews 
held with PIA principals and teachers at various times during the school 
year* Generally, all sites were visited once in the fall and once in the 
sprii»5 and about half of the interviews occurred each time* The interviews 
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with PIA teachers were focused for requisites 4 and 5 on questions about 
the lesson the teacher vas currently using. Thus, each interview concerned 
an individual lesson but the summary in Table A reflects a composite of 
lessons across the four ijrade levels. Since the interviews spanned the 
entire school year, a representative sample of lessons was assured. Addi- 
tional information to corroborate the verbal statements of the teachers 
regarding their instructional procedures (requisite 4) was secured through 
the classroom observations which occurred on the sjone day as the interviews 
for about one-half of the teachers. 

As a whole, the set of requisites was satisfactorily fulfilled in all 
sites except urban site 3. Attendance of PIA teachers at the preprogram 
inservice meetings (requisite 1) was very good in all sites except urban 
site 3. Program utlliaation (requisite 2), the extent to which the PIA 
telecasts and student exercises were used, was comprehensive in three of 
the five sites; in urban site 2 almost one- third of the teachers did not 
complete the telecasts and workbooks, and participation was totally lacking 
in urban site 3, This factor alone, obviously, makes data from urban site 2 
open to some suspicion and data from urban site 3 unusable. Requisite 4, 
which deals with the recommended instructional procedures for PIA, was met 
by the great majority of teachers, although in urban site 2 there were again 
some distinct weaknesses. All sites were notably lacking in their use of 
common mathematics manipulative aids specified by the program. 

Except for poor reception, which was a fairly common complaint, the 
other requlremen ? of requisite 3 regarding technical aspects were easily 
met in four sites. In urban site 3, however, an error in Grade 1 schedul- 
ing resulted in no schools meeting the requisite. The time allotted to PIA 
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(requisite 6) was sufficleut in four Bltes{ about one-third of the urban 
site 2 schools did not spend enough time on PIA lessons* 

The appropriateness of PIA for particular groups of students (requi- 
site 5) was negatively viewed by from one-half to one-thlrd of the teachers 
in all sites* This problem is in part due to the nature of a televised 
Instructional course which allows no variation in the pacing or order of 
lessons presented. (To alleviate these difficulties ^ the Katlonal Instruc- 
tional Television Center,"^ which distributes PIA, has since made the pro- 
gram available for audio-visual use; that is, the lessons can be shown on 
classroom video-tape players on a schedule adapted to individual student 
needs.) The results for requiisite 5 were noted when interpreting achieve- 
ment data, but sites or schools were not omitted on this basis. 

As discussed earlier, in order to be consistent with the evaluation 
plan it was decided not to eliminate data from individual schools from the 
final analysis on the basis of requisite information and to dismiss site 
data only in extraordinary circumstances. Using this criterion, all data 
from urban site 3 were omitted. Data from urban site 2 must be viewed 
cautiously in light of the requisite information. 

Portions of the requisite information were used to dprive implementa- 
tion and program a;ppropr lateness scores for each PIA teacher /class except 
those in urban site 3. These scores allow us to describe from another 
perspective the instructional practices and teacher Judgments of PIA's 
appropriateness in each site and to provide another viewpoint from which to 



^The Patterns in Arithmetic program for Grades 1-6 is published and 
distributed by the National Instructional Television Center, Box A, 
Bloomington, Indiana 47401* 
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interpret achievement results. Each PIA teacher was assigned an imple- 
» mentation score of 1-6 and an appropriateness score of 1-6 bas »d on the 
parts of requisites A and 5; both scores were very simply and arbitrarily 
derived by giving the teacher 1 point for each aspect of the two requisites 
satisfied. Some parts of requisite 4 (PIA instructional procedures) had to 
be slightly redefined in terms of a criterion; 1 point was given for each 
of the following: 

4a. The teacher had read at least three of five background segments 
of the teacher's manual prior to the telecast lesson. 

4b. The teacher had carried out a pre- telecast activity, either one 
specifically suggested in the teacher's guide or one created 
independently. 

4c. The teacher had carried out a post-telecast activity, either 
one specifically suggested in the teacher's guide or one 
created independently. 

4d. The teacher had utilized at least one "Highly Recommended" 
activity from the manual. 

4e. The teacher had used all or some of the exercise pages. 

4f . The teacher had utilized at least one-half of the common 
mathematics concrete aids at some time during the year's 
ma th ema ti c s ins t rue tion . 

An individual teacher's answers for all but 4f above reflect his utiliza- 
tion of the lesson at hand at the time of the interview. Credit was given 
only if the conditions were satisfied for all or most students, regardless 
of achievement /ability level. 

A3 mentioned above, the appropriateness of PIA for each class (requi- 
site 5) also had six aspects; most of these were judged by the teacher in 
terms of achievanent, that is, for students above, at, and oelow grade 
level, although not all three groups were represented in every class. The 
appropriateness conditions were considered met, as with the instructional 
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procedures, only if the program was viewed as suitable for all or most 

students regardless of achievement/ability level. Again, the judgments 

were made for the current lesson. An important point to be aware of here 

iB that teachers could *'plck and choose^* among the exercises and activities 

offered to adapt the program to various groups of students; this made at-* 

taininent of 5c and 5d more readily possible* However ^ such characteristics 

as the pace and order of telecast presentations could not be altered (see Sa, 

5b, 56, and 5f). Some parts of requisite 5 had to be slightly restated; 

1 point wa^ ^iven for each of the following six aspects of appropriateness: 

5a. Students should exhibit a positive attitude toward the 
telecasts « 

5b, The pace of the telecast presentation should maintain 
student attention^ 

5c. The pupil exercises should be challenging but not overly 
difficult. 

5d. Use of suppleinentary materials from other curricula should 
be largely unnecessary. 

5e* The objectives of each lesson should be challenging but 
not overly difficult. 

5f . The time intervals between telecasts should be appropriate. 

Table 5 summarizes the average Implementation and appropriateness 
scores by site and by grade. It can be seen that, in general, teachers in 
Grades 1 and 2 felt the PIA program was less appropriate for their students 
and implemented it somewhat less well than teachers in Grades 3 and 4. 
Teachers in urban areas bad lower implementation scores both as independent 
sites and as a whole than the rural sites; however, it is less easy to 
generalize about the appropriateness scores. Although overall results show 
that the program was viewed as slightly less appropriate in urban areas, 
this was due to the low ratings given by urban site 2 teachers; in urban 
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TABLE 5 

MEAN PIA IMPLEMENTATION AND APPROPRIATENESS SCORES BY SITE AND GRADE* 



Mean Implementation Scores 





Urban Site 


Urban Site 


Rural Site 


Rural Site 




Grade 


1 


2 


1 


2 


Mean** 


1 




3.79(24) 


5.23(13) 




4.30(37) 


2 




4.04(23) 


4.73(11) 




4.26(34) 


3 


4.50(36) 




4.50(8) 




4.50(44) 


A 


4.06(34) 






5.26(19) 


4.49(53) 


Mean 


4.29(70) 


3.91(47) 


4.88(32) 


5.26(19) 


4.40(168) 


Mean urban, 
rural sites 




4.14(117) 




5.02(51) 




Mean Appropri iteness Scores 


1 




2.88(24) 


3.08(13) 




2.95(37) 


2 




2.52(23) 


3.64(11) 




2.88(34) 


3 


3.33(36) 




3.00(8) 




3.27(44) 


4 


3.35(34) 






2.95(19: 


3.21(53) 


Mean 


3.34(70) 


2.70(47) 


3.25(32) 


2.95(19) 


3.10(168) 


Mean urban, 
rural sites 




3.09(117) 




3.14(51) 





^ Highest possible score is 6; scores were derived from interviews with teachers. 
^ It should be noted that the number of teachers per site is disproportionate. 
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site 1 the program was described as relatively more appropriate than in 
either rural site. An Interesting contrast appears in the results for 
rural site 2, where teachers ittplemented the program at a very effective 
level but nevertheless seemed to view it as somewhat inappropriate. Con- 
sistent with other monitoring findings in urban site 2, PIA was viewed as 
quite inappropriate and also was implemented relatively ineffectively. 
Further discussion of the relationship of these scores to pupil achievement 
appears in Chapter IV. 

Comparative Description of Field Test Schools 

The second type of monitoring Inforroatlon, summarized In Table 6, in- 
cludes both experimental and control groups and affords us a more complete 
description of the field test sites as a whole as well as a comparative view 
of the IIA and control schools in each site. Data were drawn from the in- 
struments described in Chapter II. 

Perusal of Table 6 reveals that for the characteristics examined, the 
experimental and control groups in each site and across all sites were 
generally quite comparable; however, there are a few noticeable differences. 
Fron a socioeconomic standpoint, the control groups in both urban sites and 
one rural site had a definite advantage. This can be partially explained 
in the case of urban site 2 in which assignment was not random; of neces- 
sity, a control district was matched as closely as possible to the experi- 
mental district which had the broadcast facilities. At least two schools > 
in the control district had very few disadvantaged students; all schools In 
the experimental district had a heavy majorlcy of students from Impoverished 
home backgrounds. For the rural sites, the socioeconomic imbalance can be 
partially explained by the fact that after the random assignment was 
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coi^leted^ technical facilities for television vere found to be unavailable 
in a few Qchools* These achools were then dropped from the experimental 
group and new schools were added* Typically » these schools had attained 
their telecast facilities through eligibility for special funds for equip- 
ment^ and consequently! these schools also had a greater preponderance of 
students of low socioeconomic status • Two other school-wide characteris- 
tics » aide availability and degree of previous involvement in Innovative 
mathematics programs ^ were about the same for experimental and control 
groups in four sites; there were some discrepancies in aide availability 
for urban site 2 due to the factors just mentionedi. 

Grade by grade^ the time officially allotted to mathematics instruc- 
tion as reported by the principals was roughly equivalent for the two groups 
across all sites. When teachers reported their actual instructional time 
for mathematics this equivalence was borne out across all sites ^ although at 
particular grade levels within some sites there was as much as ten minutes 
per day difference in time for mathematics • 

When ask^d to assign their students to three general achievement 
groups^ teachers indicated about the same proportions of students for each 
group in the urban sites; in the two rural sites, noticeably fewer children 
were judged to be above grade level in control schools than in PIA schools 
and more were judged to be at grade levels Taking all sites together, how- 
ever, proportions were about the same for the two groups^ 

The average number of students per teacher was somewhat less for PIA 
than for control classes in all but urban site 1. Th.e evidence of individ- 
ualization varied Inconsistently for PIA and control groups within sites, 
while use of mathematics manipulatlves was more nearly equivalent. 



In summary, the experimfental and control groups across all sites were 
generally quite comparable except for socioeconomic status. Within sites » 
urban site 2 had several conspicuous differences between control and exper> 
Imental groups » while &11 other sites showed less extensive differences. 
Subsequent differences In achievement, reported in Chapter IV, could not be 
directly attributed to lack of equivalent experimental and control groups 
in any site, although the discrepancies were kept in mind. 

An additional iton of Interest derived from Interview results is that 
in control schools, which used whatevo.r mathematics curricula were pre- 
f erred, 11 different major basic text series were represented. 



IV 

ANALYSIS OF DATA 



Introduction 

Theory 

The general purpose of the study was to evaluate the effectiveness of 
the PIA program In urban and rural sites « The specific objective of the 
study was to determine, for each grade level included, if the PIA program 
had a positive effect on mathematics achievement for the specific target 
population in the study. To achieve this objective, the design of the study 
included the random assignment of approximately equal numbers of schools 
within each of the four sites to each of two treatment conditions; about 
half of the schools were to use PIA in Grades 1-4 and about half were to use 
any program except PIA at those grade levels, prestimably whatever program 
they would have used had there been no study. The objective can be stated 
as a null test hypothesis; if students in the target population use PIA, 
there will be no effect on their achievement of mathematics concepts and 
computational skills and problem-solving skills • 

Students in all schools were tested on selected mathematics tests early 
in the academic year to provide baseline (pretest) measurements and again near 
the end of the academic year to provide data on achievement after one full 
year with whatever program they were using (l^e., posttest scores). Thus, 
within each site the mean posttest scores for the PIA schools provided un- 
biased estimates of mean performance for the schools in the site with the 
PIA program. The crrresponding mean scores for the control schools provided 
unbiased estimates of the mean performance expected for all schools if there 
had been no study. If PIA had no effect, then the two sets of estimates 
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should be ur.biased estimates of the same quantities and should differ only 
because of sampling and measurement errors • Thus, rather simple assump- 
tions about the probability distributions of the differences, e*g*^ those 
which are based on the presumed random assignment of schools to treatments^ 
and usual normal theory assumptions about measurement error distributions 
provide the basis for determining the likelihood of differences as large or 
larger than those observed* 

A positive effect of PIA for a given grade level was inferred only if 
the probability of a difference as large or larger than that observed for 
that grade was equal to or less than *05» and then only if the differences 
consistently "favored*^ the PIA schools^ The analysis for each grade level 
was done independently. 

Measurement of Mathematics Achievement 

Mathematics achievement level was a multivariate variable in this 
analysis* The measure used has three basic components: (1) a measure of 
ccncept attainment trr \ standardized tests of mathematics concepts appro- 
priate to the grade level at which it is used; (2) a measure of computa- 
tional proficiency also from standardized tests appropriate at each grade 
level; and (3) a measure of proficiency in application of concepts and 
computational skills to problems from standardized tests appropriate for the 
grade level. These three components are hereafter termed concepts ^ computa- 
tions , and pr o blems * The analysis used multivariable analysis of variance 
to estimate program effects and to test the various statistical hypotheses 
required for Inferences* Only one of tiie two icst scores available for con- 
cepts was used in each case because it was felt that the two scores avail- 
able were so highly correlated that the use of both would not increase the 
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information gained enough to coiRpensate for the loss of degrees of freedom 
that vould resxilt* However^ univariate results are presented for both 
tests % 

In addition, several subscale scores for the concepts and computations 
components were also analyzed to supplement the main analysis » These were 
constructed to be measures of specific content areas (numbers and numeralsi 
addition, etc.)- Moreover^ the relevant items which are specifically con- 
tent valid for PIA were separated from those which are not to form PIA 
content valid as well as nonvalid subscales (see Appendix D) . Thus, the 
analysis of these scores yielded information regarding the effectiveness of 
VIA specifically in relation to its obj actives n The final inferences or 
conclusions regarding the overall effectiveness of PIA were based on the 
most reasonable interpretation of all the analytic results. 

Unit of Analysis 

Students were assigned to experimental conditions by school; hence, 
assignment of students was not by simple random process. Because of this, 
the statistical unit of analysis within ^^ach grade level was the school 
rather than the student* However, this unit is also the more theoretl- 
cally relevant as the program is applied on a school (or a classroom) basis 
and it should follow that one would want to determine effectiveness on per- 
formance for the school (or classroom). Unfortunately, as inmost studies 
which use the student as the unit of analysis, attention was focused on mean 
performance levels for each school within each grade level rather than on 
the entire performance distribution for each school. The mean level is but 
one aspect, and it is an oversimplification to take it as the only or even 
the most relevant aspect of that distribution* However, it is analytically 
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convenient to do so^ and when the number of schools is small ^ as in this 
study > it is difficuli if not impossible to perform valid statistical 
analyses with multiple measures of the entire distribution « 

However, so as not to ignore all variation from the mean entirely > two 
mean scores were computed for each school on each test administered « StU"- 
dents were categorized by sex, and a mean score for each group was computed » 
Sex was chosen because the data identifying sex of students were available 
and because there is some evidence that mathematics achievement does depend 
on sex* 

These two scores » one per sex group, were treated as repeated measures 
on each school for each achievement component considered* For this study, 
there was little interest in the ^Win effect" of sex on mathematics 
achievement* The key question was whether the effectiveness of PIA varies 
with the sex of the student, l«e«, whether there was a "treatment by sex 
interaction." If such an interaction was inferred, then the main effect of 
PIA-— the average of the two sex-specific ef f ects*-*-would not be used as the 
criterion of effectiveness unless the sex-specific effects were In the same 
direction, e^g*, unless it was evident that PIA effectiveness varied only in 
magnitude but not direction with sex. Otherwise, If the sex-specific 
effects varied greatly in magnitude, the effectiveness of PIA would be ana- 
ly^ed specifically for each sex group. If the interaction was inferred, the 
main effect of PIA might be no more meaningful (for program planning) than 
telling someone that a river can be crossed on foot because its average 
depth is only three feet. If the sex-specific effects did not appear to 
differ by more than a "small" magnitude, the average or main effect would 
be used as the criterion, (""c-^^jli" in this case indicates too small to de- 
tect by the statistical tool - . interaction.) 
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?relimlTiary Test on Site by Treatment Interaction 

Schools in two sites— one urban and one rural— were studied at each 
grade levels There was thus the possibility that PIA effectiveness might be 
fo^nd to vary with site either in general or by sex of student. The main 
effect of PIA was the average of the two site-specific effects^ and this was 
to be used as the criterion of effectiveness only if site differences in PIA 
effectiveness appeared to be small or if the site-specific effects were not 
greatly different and in the same direction. 

Thus^ two preliminary tests were performed at each grade level. The 
first was to determine If PIA effectiveness varied with sex in either site^ 
l%e.» a test of sex x treatment combined with sex x treatment x site inter- 
action. If the test statistic was not significant, the second preliminary 
test would be used to determine if PIA eff ectlven^jsss varied with site, l*e., 
to test the hypothesis of no site x treatment interaction. The signifi- 
cance level used for both tests was the .15 level to insure reasonably ade- 
quate power for detection of lack of fit to the simple additive model while 
not unduly raising the risk of using a more complicated model when the 
simple one would be adequate for the purposes of this study. 

Implementation Factors 

Inferences concerning program effectiveness are limited to the results 
for the program as implemented for the populations urider study. The levels 
of implementation might be less than optimal (as that is an ideal) but per- 
haps no better or worse than what might be expected should the program be 
adopted In these or comparable sites. However, Implementation of any pro- 
gram must, at least a priori , be considered an Important factor in realized 
effectiveness. Whether good implementation or bad implementation can 



38 



account for results such as those to be reported is difficult if not im- 
possible to determine » The probli^ is that for experimetital purposes the 
objective is usually to control implementation at or near an optimal level 
while the actual levels achieved are in part determined by factors beyond 
experimental control which may directly or indirectly in other ways affect 
achievement levels. This does not invalidate statistical inferences, but it 
can reduce the power of detecting true program effects and affect the prac- 
tical significances of the inferences • 

The available data could not yield an answer to the question of how 
much the results, whether positive or negative^ were influenced by the level 
of implementation^ However, the influence of this factor was considered 
worthy of some exploration as was the factor of the appropriateness of the 
PIA material to the students in the study ^ To study the effects of imple- 
mentation^ each PIA classroom was given a rating designating the degree -o 
which a qualified Center observer considered the PIA program to have been 
implemented relative to instructions given all PIA teachers during in- 
service. The scale ranged from 1 for the lowest level of implementation to 
6 for the highest level. Mean scores by grade were computed, and the cor- 
relations v:f this score with the three main components of mathematics scores 
were computed an3 analyzed. To study appropriateness of the material for 
the students, a measure of appropriateness was similarly derived, based on 
responses of PIA teachers to questions about the extent to which they per- 
ceived the PIA materials appropriate for their studenf:s. These data were 
analyzed in the same manner as the implementation scores. 

Obviously, such results suffer from the well-known defects of corre- 
lational analyses. One can not be certain of what other variables may be 
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correlated with either of the two scores just described* Moreover^ there 
is no assurance that variation in either implementation levels or levels of 
perceived appropriateness is great enough to permit reliable conclusions* 
But, if interpreted with caution* such analyses may provide a valuable 
perspective for interpretation of the results of the main analysis* 

Missing Data 

In some cases students present for fall testing were absent at spring 
testing and vice versa* This presented no problem as the school (within 
grade level) is the unit of analysis and mean scores of students present for 
each testing provided unbiased estimates of mean achievement levels for the 
school/grade at that point in time. 

In a small number of other cases, data on one complete instrument were 
missing for the entire school/grade for either fall or spring testing (but 
not both) but data on remaining instruments were available. This usually 
resulted from misunderstanding of directions or from administration of the 
wrong test. In these cases estimates of the missing scores were obtained 
using a ratio-estimate method. For example, suppose the fall score on com- 
putations was missing for Grade 1. The mean for the remaining Grade 1 
schools on fall computation was computed. In addition, the Grade 2 fall 
score on computation was available as well as the mean score for all other 
Grade 2 schools for fall. Tl^ estimate of the Grade 1 score was then made 
equal to the mean of all other Grade 1 schools multiplied by the ratio of 
the Grade 2 score for that school to the mean Grade 2 score for all other 
schools. 

Such estimation vas required in 12 of the total 912 cases (each case 
is a pretest or post test score for both sex groups on one main measure for 
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one grade level for one school) « In three cases posttest (spring) scores 
were estimated , and in nine cases pretest (fall) scores were estimated. Ko 
posttests were estimated for PIA schools, as all had complete spring data. 
Pretest scores were estimated for four PXA and five control schools « 

Of less importance, in two cases (one PIA and two control) no scores 
for girls were available on pretests . These cases are for the same grade 
level (Grade 4) in the same site (urban site 1)» In these cases the sex 
difference was taken to be aero. (Dropping these two schools from the 
analysis of the sex by treatment produced only minor changes in results*) 

Use of Analysis of Covariance 

As might be expected, the pretest scores indicated great variability 
among schools from the same site* This variability was the result of a 
large number of mostly unknown factors which, it seemed most likely, would 
produce comparable variation in posttest scores within each sample (PIA or 
control) for each site* With random assignment, the effects of this vari- 
ability could be treated as "error^' variance and assumptions concerning the 
probability distribution of this and other sources of experimental error 
could be made. However, if this variability was great the power of statis* 
tical tests might have been greatly reduced* Analysis of covariance was 
used here to increase the power by reducing the magnitude of the extraneous 
variation attributed to error* Pretest scores were selected as candidates 
for covarlates. First, the regression of posttest scores on pretest scores 
was checked to determine if the observed (linear) relationships were statis- 
tically significant and also to determine if the residual error variance 
after "removal^' of the covariates had been reduced* In all cases, the hypo- 
thesis of no relationship was rejected and considerable reduction In error 
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variance was found. Hovever, results of analyses of variance both with and 
without covariates are shown. 

In urban site 2 and for a few schools in other sites, random assignment 
was not achieved. In this case the efficiency of analysis of covariance 
might have been seriously reduced if the actual assignment tended to favor 
one of the experimental conditions over the other. This would result from 
the effective confounding of the covariates with the treatment factor which, 
while perhaps reducing total er-'or variance, would tend to greatly increase 
the variances of treatment effect estimators. However, in this case the 
essential problem was whether the usual probability distributional assump- 
tions were justified in the absence of random assignment even with the ad- 
justment for covariates. It was presumed that this was the case merely to 
present some analysis. However, the reader is advised to keep this con- 
sideration in mind. 

Results 

Preliminary Test Results 

Results regarding site x treatment and sex x treatment x site inter- 
actions are substantially the same for all four grade levels: whatever the 
effect of PIA, it appears to be substantially the same for each site and 
both sex groups. The results of the statistical tests using (multivariate) 
analyses of variance are summarized in Tables 7-10 for the three major com- 
ponents of mathematics achievement. Results for the subsca^es formed of PIA 
content valid Items were generally the same and are thus not presented. 
Sample mean scores by grade, by site, and by experimental group on both pre- 
tests and post tests. Including the subscales, are shown In Tables 11-14. 
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Treattnent Effect 

Estimates of treatment effect^ i*e*, estimates of the difference be-- 
tween treatment and control ^'true** means^ obtalTied by the method of least 
squares assuming no variation in PIA effect by site or sex are shown by 
grade in Tables 15-18* Estimates of standard errors are also shown. In no 
case can the null hypothesis of no treatment effect be rejected using the 
criterion that the I^-ratio approximation to the multivariate test be signi- 
ficant at the .05 level. While only results for the major components of 
mathematics achievement are shown in the analysis of variance suimnary tables 
(Tables 7 to 10), results for the subscales are generally quite similar* 
For Grades 1, 3, and A the evidence against the null hypothesis, as measured 
Inversely by the observed significance level of the test statistic, can not 
even be considered moderately strongs For Grade 2 the significance level 
observed is ^07; however, the tests used are non-directional and, if any- 
thing, this evidence would point to a difference in favor of the control 
schools* This is indicated by the negative estimates of treatment effects 
shown in Table 16. 

In fact, with very few exceptions the estimated PIA effect is negative 
in all grade levels and both s^ groups » The exceptions are for the prob- 
lems subtest, two subscales for Grade 4, and three subscales for Grade 3, 
However, only in Grade 2 are the estimated values substantially larger than 
the corresponding standard errors of the estimates. In Grade 2 the esti- 
mates are almost uniformly at least double the size of the corresponding 
standard errors and in most cases would be statistically significant by 
univariate .05 level two- tailed t-tests. For all other grade levels, the 
most that can be inferred is that the program produces little change in 
achievement over existing programs • 
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Galn» or more neutrally, change^ scores obtained by subtracting pretest 
scores from posttest scores are available for all grades except Grade 1, 
Estimates of the effect of PIA on change during the academic year are also 
shown in Tables 16 to 18, In this case^ with few exceptions, the effect is 
positive, i.e,, students in PIA schools had greater observed gains than 
those in control schools* However, the multivariate analysis of the gain 
score data (not shown) does not provide even moderately strong evidence 
against the hypothesis of no PIA effect* 

Pretest Blfferences 

Part of the discrepancy between the observed differences for posttest 
scores and those for pretest scores can be attributed to the fact that pre- 
test scores were generally higher for control than for PIA schools (Tables 
11-14). This situation was almost certainly caused by the inability to 
maintain stringent random assignment, particularly in urban site 2 as ex- 
plained previously. In one case (Grade 2), the difference between treatment 
and control sample means is statistically significant using the same cri- 
terion used for posttestB* Moreover, pretest and posttest scores are cor- 
related to a statistically significant extent as shown in Tables 7 to 10» 
In addition, by comparing the mean squares for error without and with the 
covariates removed in Tables 7 to 10^ one can appreciate the extent of the 
association. Except for computations, the mean square for error is reduced 
by about 50% for the variables at all grade levels • The decrease for compu- 
tations is about 33%. Given these results, it is not surprising that post- 
test scores are higher for control schools. 
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Analysis of Covariance 

The estimates of the effect of PIA on mathematics achievement after 
removal of covariates are shown in Tables 15 to 18. For Grades 1, 3, and A 
the use of covariates produces moderate to large decreases in the standard 
error of the estimates of the PIA effects. This is not true for Grade 2 
largely because of confounding between the treatment "factor" and the co- 
variates. For Grades 1 and 4 the adjusted estimates for the PIA effect on 
the major components of mathematics achievement are positive. For Grades 
2 and 3 the adjusted estimates are negative with one exception for Grade 3 
but are much smaller in absolute magnitude than the unadjusted estimates. 
However, the analysis of covariance results do not provide even moderate 
evidence against the hypothesis of no PIA effect. 

Summary of Results of Analyses 

The results of these analyses do not provide sufficient evidence to 
reject the hypothesis of no effect of PIA on mathematics achievement in favor 
of the hypothesis of a positive effect at any grade level. Rather, the evi- 
dence suggests that student achievement levels are about as high with pro- 
grams generally in use in the target populaticns studied as under PIA. These 
results appear to be the same regardless of sex of student or whether the 
school is in an urban or in a rural setting. 

Implemeptation and Appropriateness Results 

Mean scores and variances for level of implementation for the PIA 
schools are shown in Table 19 by grade. The correlations of level of imple- 
mentation with each of the three main measured of mathematics achievement are 
also presented in Table 19. It Is somewhat puzzling that in Grades 1 and 2 
the observed correlations are negative; however, none of these differ from 
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zero by a statistically significant amount. Of course, the small number of 
degrees of freedom for error makes both estimation and testing quite impre- 
cise at these grade levels. For Grade 3 the correlations are positive and 
the relationships are statistically significant. (The multivariate test of 
the regression of the three main components of mathanatlcs achievement o\i 
Implementation produces a result significant at the .05 level.) Moreover, 
the variability of Implementation at Grade 3 appears to be considerably 
higher than at Grades 1 or 2. For Grade 4 the observed correlations are 
positive but smaller in magnitude than those for Grade 3, perhaps partly be- 
cause of the somewhat smaller variance of implementation scores. The ob- 
served correlations at Grade 4 are not statistically significant. 

An analysis paralleling the one just described was performed on scores 
obtained from teachers* responses to questions designed to determine the 
extent they perceived the materials in PIA appropriate for their students. 
Means and variances of these scores as well as observed correlations with the 
main measures of mathematics achievement are shown in Table 19. In general, 
the correlations are small. For the most part, they are negative for Grades 
1 and 2 and positive for Grades 3 and 4. None are statistically significant. 



TABLE 19 



IMPLEMENTATION OF PROGRAM AND APPROPRIATENESS OF MATERIAL— 
CORREUTIONS WITH MATHEMATICS ACHIEVEMENT OF PIA STUDENTS 



Score 


1 


GRADE (PIA only) 
2 3 


4 


Implementation score 
Mean 
Variance 


4.46 
.71 


4.25 
.59 


4.49 
1.57 


4.42 
.93 


Appropriateness score 
Mean 
Variance 


2.96 
1.16 


2.98 
.74 


3.30 
1.87 


3.25 
1.38 


Correlations 

Implamentation with 
Concepts (ETS) 
Concepts (CAT)® 
Computation (CAT) 
Problems (CAT) 


-.37 
-.31 
-.42 
-.36 


-.49 
-.60^ 
-.44 
-.52 


.37* 
.50^ 
.38® 
.56^ 


.27 
.30 
.18 


Correlations 

Appropriateness with 
Concepts (ETS) 
Concepts (CAT) 
Computation (CAT) 
Problems (CAT) 


-.12 
-.09 
-.09 
-.31 


.06 
-.02 

.11 
-.19 


-.05 

.03 
.15 
.14 


.01 
.11 
.10 


df error 


8 


8 


24 


27 



® Significant at .10 level. 

^ At Grade 4 the CTBS was administered instead of the CAT. 
^ Significant at .05 level. 



SUMMARY AND DISCUSSION 

The central question of this evaluatioTi was what the effect of the PIA 
program was on student achievement in mathematics in Grades 1 to 4. The 
schools selected for the study represented urban and rural settings of gener- 
ally low socioeconomic status in whiv-h a television-based mathematics program 
thought to have particular advantages, e.g., requiring less advanced reading 
skills. The results of this one-year study indicated, however, that pupils 
in control schools who participated in several conventional mathematics pro- 
grams achieved equally as well as the PIA students on standardized measures 
of achievement in mathematics concepts, skills, and problem solving. These 
results did not vary with sex, site, or grade level. 

Apparently, the possible advantages of television instruction did not 
in this case outweigh the disadvantages frequently pointed out by the parti- 
cipating teachers. A severe drawback present when television lessons are not 
available in video-tape format is the rigidity of a broadcast schedule that 
allows neither for flexibility of lesson presentations within the daily or 
weekly classroom schedule nor for individualization in pacing the lessons for 
students of different ability. (PIA is now available in video-tape format.) 
<?;nilarly, ther-2 is no possibility of varying the rate at which a single 
telecast is pre' '"^ ted or of pausing during instruction to clear up questions 
or repeat segments. In general, the lack of interaction between television 
teacher and student — the impersonality of such instruction— is seen as a 
disadvantage. For the PIA telecasts in particular, the fact that they were 
not produced in color or with animation makes them less appealing to students 
accustomed to commercially produced programs and high-budget shows such as 
"Sesame Street." 
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A principal feature of televised mathematical instruction is its use of 
manipulative and visual aids to illustrate abstract ideas; it is assumed that 
teachers will model their follov-up instruction on the telecast presentation 
by relying heavily on manipulative objects. Tield test monitoring visits 
verified that appropriate manipulatives were readily available in the experi- 
mental classrooms but that less than half of the teachers ever used them* 
Thus, many teachers depended completely on the telecast presentations for 
translating the mathematical abstractions into realistic situations for their 
students. A more extensive inservice training period is probably necessary 
to equip teachers to employ televised instruction effectively • 
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The following is a brief description of the fundamental concepts which 
run through the Patterns in Ar^ithmeHa series. 

1* Set ; The concept of a set is fundamental for developing and cotnmuni*- 
cating ideas in mathematics* Since the idea of a set is a rather 
primitive one^ as early as Grade 1 pupils become familiar with the 
simple concepts which involve sets, such as setting up a one-to-one 
correspondence (matching) between sets and comparing the numerousness 
of two sets* The set idea is found to be particularly useful in the 
teaching of addition (set union) and subtraction (set complementation). 
Set ideas are also applied in the later grades to equivalent fractions, 
equivalent ratios^ and geometric figures. 

2. Number: One of the main features of the program is the logical devel- 
opment of the rational numbers^ beginning with the positive integers. 
Near the beginning of Grade 3 the pupils will be able to determine the 
cardinality of any set with less than 10,000 members and to use the 
ordinal numbers. During the First, and again the Second Grade, the 
pupil is taught the ideas of betweenness , less than, greater than , and 
equal to for niimbers less than 1,000. The number line is used to pic- 
ture the order of integers and the v^pil is taught how to use this line 
to perform addition and subtraction. These ideas are reinforced and 
extended so that near the end of Grade 6 the pupil is totally familiar 
with the positive rational numbers. Other miscellaneous topics covered 
in the series are even and odd numbers, prime numbers, prime factori- 
zation of natural numbers, the least common multiple and greatest com- 
mon factor of pairs of natural numbers, and the negative integers. 

3. Numeration System : After some degree of understanding has been 
achieved in counting with both cardinal and ordinal numbers, the pupils 
are taught how to write the numbers 0-9, and after 9, the concept of 
place value is introduced* Although the pupils do not fully comprehend 
place value at this time, extensive use of tally charts and regrouping 
is Introduced in order to overcome some of the mystery of place value. 
Near the beginning of Grade 4 the pupil should be able to write all 
numerals and interpret place value for larger numbers. Although empha- 
sis is on the decimal system (Base 10), numeration systems with other 
bases (Base 5, Base 2) are introduced to proi te understanding of place 
value. Positive rational numbers and the several notations associated 
with them (common fraction^ mixed number, decimal fraction) are intro- 
duced in Grades 4-6. 

4. O pe ra tions ; A considerable portion of elementary school arithmetic is 
concerned with the four fundamental operations—addition, subtraction, 
multiplication, and division — and how these operations act with vari- 
ous sets of numbers (natural numbers, positive ratlonals, integers). 
Of course, one of the major objecfives of any elementary school arith- 
metic program is to develop accuracy and ^^peed in computing. However, 
computing in itself Is not enough; thereiore, we have placed consider- 
able emphasis on ideas associated with computing in an attempt to make 
the four operations more than rote calculation. For example, in form- 
ing the sum 3 + 2, the pupil considers a stationary set with 3 objects 



and another set of 2 objects which appear to be joining the given 3. 
Thus 3 + 2 is looked upon as a set of 3 being joined by a set of 2 to 
form a set of 5^ The concept of colxmn addition is approached through 
tally charts in an attempt to impart some understanding of place value 
in our Base 10 numeration systean. Rectangular arrays are used to 
analyze geometrically the operation of multiplication* Geometric ar- 
guments are made to lend substance to the operations applied to the 
positive rationals. The notion of operation Is extended in Grades 5 
and 6 by introduction of an operation with a simple geometric reali- 
zation^ 

Beginning in the First Grade the commutative, associative, and 
distributive properties of the operations receive considerable atten- 
tion as aids to computation. Hot until Grade 5 are the properties 
formalized for the pupils. Upon completion of Grade 6 the pupil will 
be able to compute efficiently with the positive rational numbers in 
their many notations. 

Throughout PIA the pupils are presented verbal situatiuus in which 
they can apply their newly acquired computational skills* In these 
situations special attention is devoted to developing the ability to 
formulate mathematical sentences in a clear and natural way. 

Mathematical Sentence (Equation): Since language is the vehicle 
through which we communicate our ideas, it is important that pupils 
begin to develop the ability to generate mathematica', sentences as soon 
as possible. In the Pirst Grade, the pupil encounters many experiences 
with pictures and objects which lead to basic sentence forms. In word 
problems pupils are requested to ^Vrite a sentence which tells the 
story of the problem*' before finding the solution. (Problem: A box 
holds 6 apples. How many boxes are needed for 30 apples? Sentence: 
N X 6 = 30.) Throughout the series the same importance is attached to 
the need to translate verbal problems into mathemi=»tical sentences^ 

MeasurCTient : Owing to its importance in everyday life as ^ key link 
between our physical and social environment, we begin a systematic 
study of measurement in the First Grade. As an Introduction, the First 
Grade pupil becomes familiar with the concept of relative length (the 
desk is less than five pencil-- lengths long) and a non-standard unit of 
measurement (the pencil above). Upon completing the first four grades, 
the pupil will be able to carry out approximate linear measurements in 
standard units (inches, feet) and he will be able to find the perimeter 
of some elementary geometrical forms (triangle, rectangle). In Grades 
5 and 6 the concepts of area and volume are discussed in terms of non- 
standard and standard units of measure and approximation techniques are 
used to estimate areas and volumes of irregular regions. Formulas for 
areas or volumes of elementary regions, such as a rectangular region 
and right rectangular prisms, are introduced. 

Geometry : One of the more unique features of the program is a system- 
atic development of elementary geometrical concepts beginning in the 
First Grade. Aside from learning the names of the more common geo- 
metrical figures, the pupil becomes familiar with open and closed 



curves, interior and exterior of geometrical forms, points, lines and 
angles, intersection of curves, parallelism, and perpendicularity. The 
notions of congruence and similarity are introduced intuitively in the 
early grades and in the later grades approached from transformations in 
the plane (reflections and dilations). 

Practical Aspects ; Although the practical aspects should not perhaps 
be mentioned as a fundamental strand, these aspects are important 
enough in the daily activities of pupils to warrant special attention. 
Beginning in the primary grades the pupil is exposed to aspects of 
linear and cubic (cup, pint, etc.) measurement. By the end of Grade A 
the pupil will have some experience in measuring the boundary and area 
of plane geometrical forms. Other practical aspects covered in the 
first three grades are money and iUaking change and use of the thermo- 
meter. 
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MEMORANDUM OF AGREEMENT 



The Wisconsin Research & Development Center for Cognitive Learning 

and the agree cooperatively to 

field test> during the 1970-71 academic year, the Patterns in Arithmetic 
instruGtional materials developed by the Center. 

The Center will provide: 

1, All materials for students and teachers including one teacher's 
manual for each experimental teacher , workbooks for all students 
in the experimental schools , and video materials including the 
related shipping costs* In regard to the video materials the 
Center will either pay rental charges for video tape to be 

broadcast by the or an 

equivalent amount if the prefers to acquire video tape on 

a more permanent basis, 

2, Spring and fall inservice workshops for all teachers of experimental 
and control classrooms* The Center will pay each teacher and a 
maximum of five additional staff honoraria at the rate of ^ per 
day. Staff and consultants to conduct the workshops will be fur- 
nished by the Center. 

3* A field test coordinator who will be responsible for the conduct of 
the field lest and field test monitors who will visit each experi- 
mental and control school at least once during the year. 

4, All evaluation instrtjments and related scoring services. 

5. Copies of a final report showing the results of the field test. 
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The Willi 

1. Arrange for video reception facilities and classroom reception 

time. 

2. Provide essential supplementary instructional materials in the 
experimental schools as recommended in the PIA manuals. 

3. Provide the usual program of arithmetic instruction in the control 
schools. 

4. Permit random selection and assignment of schools to experimental 
and control conditions. 

5. Designate one staff member from the 

as a field test coordinator and one staff member in each school as 
a local coordinator. These coordinators will monitor the day to 
day conduct of the field test and make student records and data 
available to the Center. 

6. Provide facilities and release staff for the inservice workshops. 

7. Follow field test procedures recommended hy the Center. 



SIGNED 5 



Research & Development Center 



8/17/70 



APPENDIX C 
INSERVIGE AGENDAS AND MATERIALS 



Sample Agenda 
Orientation Meeting 
for the 

Patterns in Arittoietic field test 
September 19, 1970 

Welcome and Introductions 

Overview of the National Field Test 
Sites 

Objectives 
Testing Program for Students 
Teacher Questionnaire and Test 
Lunch 



Patterns in Arithmetic Utilization 
(for PIA teachers only) 



Sample Agenda 
Final MeBtlng 
for the 

Patteirns in Arithmetic field test 
May 1, 1971 

Welcome and IntrDductlons 

Field Test Results 

Interviews and Observations 

Teacher and Student Test Performance 

Final Report 

Spring Testing Program for Students 

PIA Programming for 1971-72 

Teacher Questionnaire and Test 
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Outline of PIA Major Topics 

Set 
Number 
Numeration Systems 
Operation 

Mathematical Sentence (Equations and Inequalities) 

Measurement 
Geometry 

Number Theory 
Practical Aspects (Time, Money, etc.) 
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OUTLINE OF PIA MANUAL 



THE STUDENT CAN 

• Behavioral objectives 

THE TV TEACHER WILL 

• Telecast synopsis 

OVERVIEW 

Lesson synopsis 

MATHEMATICAL BACKGROUND 

Mathematics content 
Purpose of the lesson 
Comments 

-^PRE-TELECAST ACTIVITIES 

^ir"MATERIALS 

During the telecast 

TELECAST DESCRIPTION 
^ POST-TELECAST ACTIVITIES 

FOLLOW-UP ACTIVITIES 

Jjc Jjc JfCHighly Recommended 
% ^ Optional 
:{( Advanced 

DIRECTIONS FOR PUPIL EXERCISES 
CHECK-UP EXERCISES 
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SUMMARY OF SaBSGALES FOMIED FROM 
PIA CONTENT VALID ITEMS 
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APPENDIX E 

MONITORING INSTROMENTS: INTERVIEW 
AND OBSERVATION GUIDES 
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interview Guide 
Teachers in PIA Schools 



Teacher^ 
Grade 



School 



Interviewer 



Directions: The purposes of this interview are (1) to determine specifically 
how one PIA lesson was used by the teacher and (2) to identify 
attitudes of teachers and students toward one TIA lesson^ 

After establishing rapport^ explain that the information given 
is confidential and will be pooled with that from interviews 
on other dates and in other sites. Suggest that you wish to 
talk about the lesson finished iimnediately prior to the current 
one and If the teacher has his manual ^ ask him to locate this 
lesson and open your own manual and exercise book to it* 
Pointing to the relevant sections ^ ask the following questions 
giving the teacher time enough to skim the manual when necessary, 
(You might want to make a brief introductory statement or two 
about the content of the telecast and lesson to help the teacher 
recall it/) 
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Program If 



Which of the following sectioi^s did you actually take time to read before 
the telecast? 



Grade 3 only Grade 4 only 

The Student Can Objectives 

The TV Teacher Will GoTmnents 

Overview Suggested Activities 

Mathematical Background 

Telecast Description 

Did you actually use this suggested Pre-telecast Activity with your class?** 
Ot did you make up one? 

Yes » used this one Yes , created one No^ did not use any 

What» in general^ was the attitude of your students during this telecast? 

obviously interested, indifferentj or partly obviously bored, 

enthusiastic interested, partly not restless 



other (specify) 



Rate the pace of the TV Teacher's nresentatlon during the telecast • 

about right too fast too slow 

Did you actually use this suggested Post-telecast Activity with your class?*. 
Or, did you make one up? 

Yes, used this one Yes, created one No, did not use any 



NOTE: Indicate here that the next few questions require answers according 
to the general achievement levels of the children in their class. 

(Background question) Roughly, how many of your students fall into ctach 
of these groups? 

Number of Children 



above grade level in 
general achievement 



at grade level 



below grade level 
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7. Of these sxiggested Follow-up Activities ^ how many did your students 
actually do? 

Number of Activities 

Students: Highly Recommended Optional Advanced (^) 

(Grade 3 only) 

above grade level _ 

at grade level 

below grade level ..^^^ , , 

d« Of these pupil exercises » which ones did you actually uset 

Students: All ? How Many? 

above grade level . . 

at grade level 

below grade level ...^^^ 



9i -.Rate the difficulty of the exercise pages for: 

about right too hard too easy 

Students above grade level , 

at grade level 

below grade level 



10 « Did you need to use any supplementary textbook or workbook assignments 
for this lesson? 



No Yes 



Students above grade level 
at grade level 
below grade level 

Wha^ text series? 



( ti t le J pub lish er ) 

11* Did you need to make up any worksheets for this lesson? 

No Yes 

Students above grade level 

at grade level . _ . 

below grade level 



12 • As a whole, rate the difficulty of the concepts or objectives of this 
lesson for: 

about right too hard Loo easy 

Students above grade le\el 

at grade level 

below grnde level ^^.^ , , 
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13. Bid you use the repeat telecast for any students? 

all students no students students above grade level 

students at grade level 

students below grade level 



14 » Wa3 the length of time which elapsed letween this program and the new 
telecast appropriate? 

about right no, not enough time no, there was too 

to readi ob jectives much time p . .s tudents 

of the lesson were ready to go on 

What is the average length of your arithmetic class each day? 
minutes 

16. Has using Patterns in Arithmetic made you spend more time on arithmetic 
than you would ordinarily ? 

Yes No Comment (how much additional time?) - 



17a. Which of the following manipulative materials (homemade or commercial) do 
you have available in your classroom? 

Available 

flash cards 

number line 

fraction pie 

geometric shapes (2-D or 3-D) 

counters 

rulers 

place value chart 
play money 

flannel board (or magnetic board) 

other — — 

17b. Which of the above materials have you actually used in conjunction with 
PIA this year? 




18. Have you had any probl^As utilizing Patterns in Arithmetic ? 
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IJJ . What are some of the assets of Patterns in Arithmetic? 



Comments (interviewer's general impressions): 
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Interview Guide 
Teachtirs in Control Schools 



Teacher* 

Grade 
School 

Interviewer ^^^^^ 

^Note if Mathematics teacher 
rather than homeroom teacher 

lArections: Feel free to ask these questions in any order, to reword them, 
and to ask additional questions or omit questions when appro- 
priate. Although the specific responses requested here are 
important, often the interviewer's general impressions are also 
very valuable* 

After establishing rapport with the interviewee, explain that 
the information to be given is confidential and will be pooled 
with that from interviews on other dates and in other sites so 
that no one teacher's responses will be quoted or reported 
individually. Indicate that all questions refer to the teacher's 
mathematics class, as opposed to her homeroom class (if different) ♦ 
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1. Mixat is the average, length of your arithmetic class each <lay? 
minutes 

2. Roughly, how many of your students Tall into each of these gnyupa? 

Number of Children 

above grade level in at prade level below grade level 

general achievement 

3. What basic textbook series (or other mathematics program) do you use? 
Title Publisher Edition (yr.) 



3a. If more than one basic tesctbook is used, please identify and indicate for 
which children each text is used and for what percent of instructional 
time each is used. 



A, Do you use the workbook (if any) which accompanies your basic textbook 
S'iries'i Yes No 

For all children'^ Yes No 

Specify: - — 



4a. If not, ia some other workbook used? Yes No 

Specify: . ^— 

5. On the average, how many workv'sheets (duplicated exercises or chalkboard 
exercises, etc.) do you use p^-r day (or week) for arithmetic (not in- 
cluding ase^gnments from the textbook)? 

exercises per 

6. On what page in your textbook (s) is your class currently working? 

6a. If some children are working on dlffer^ut pages (or in different texts) 
please specify: ' 
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7. How far do you expect to progress in your text series this year? 
will finish the text 



will finisli and go on to 



will not finish — Specify: 



8. Rate the difficulty of the concepts or objectives of your textbook(s) 
for; 

about right too hard too easy 

Students above grade level 

at grade level 

below grade level — — 

9. Which of the following manipulative materials (homemade or commercial) 
do you have available in your classroom? 

Available Used 

flashcards , 

______ number line ... 

fraction pie . -- 

geometric shapes (2-1) or 3-D ) ____ 

counters 

rulers 

place value chart 

play money , ,. 
fliinnelboard (or magnetic board) 

other . 

9a. Which of the above materials have you already used this year? 



10. In general, what is the attitude of each of these groups of students toward 
arithmetic? 

frustrated, bored indifferent, enthusiastic, 

neutral interested 

Students above grade level 

at gA'ade level . — 

below grade level — ^ 

Do you have access to the televised pr-^c-'wa (Patterns in Arithmetic^ ? 

Yes No 

If yes, how many of the telecasts has your class viewed this year (if any)? 

none number of telecasts (Note if all telecasts 

have been viewed) 



11. 
11a. 
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12. How do you feel about your participation in this study (as a control 
school teacher)? (e.g., do you feel you are working harder on math 
this year or spending more time than you ordinarily would?) 



13, Is there any other information about your class, of whatever kind, that 
you feel would help in Interpreting the results of the study? 



Interviewer's general impressions i 



PRINCIPAL QUESTIONNAIRE 
PIA Schools 



PrincipaJ 
School 
Date 
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1. Please estimate the percentage of children in this school 
who fall into each of these groups: 

Negro (black) White (except Puerto Mean, Cuban, 

etc*) 

Puerto Rican, Cuban, Other (specify) 

etc. 



la. About what percent of the Puerto Rican, Cuban, etc, , children 
speak primarily Spanish at home? 0-25% 26-50% 

51-75% 76-100% 



2. Can you estimate what percent of the parents of children in 

this school have a mediar annual income of below $4000 (or are 

on welfare)? 

0-25% 26-50% 51-75% 76-100% 

3. Can you establish what percent of the children in this school 
are one year or more below grade level (according to standard- 
ized tests) 

in reading? 0-25% in math? 0-25% 

26-50% 26-50% 

51-75% 51-75% 

76-100% 76-100% 
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4, How are the Grade 3 and Grade 4 mathematics classes grouped? 

Grade 3 Grade 4 

heterogenously _____ . 
hoiaogenously by general achievement 
bomogenously by math achievement ^___^_ 
homogenous ly by reading level 



5. If the Grade 3 and Grade A teachers for mathematics are not 
the homeroom teachers ^ how were they selected? 



61, Is there any auxtlilary aide or mothers' assistance available 

to Grade 3 and A teachers? Yes .. No 

If so, how many hours per week? 



7» What is the time allotment for mathematics in Grades 3 and 4 per 
day (or week) in minutes? 

Grade 3; minutes per 

Grade A: minutes per 



8, Who determines the allotment? (state, city, Archdiocese, local 
school board) 

9* How much change, if any, have you noticed in the amount of time 
spent on mathematics since Patterns in Arithmetic was started? 



more time on Math (specify) _ 

less time on Math (specify) 

no change 

10. Do you happen to know what textbook series (by publisher, edition, 
and title, if possible) was used last year for 



Grade ^ 
Grade A 
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11. In the last two years, has your school been involved in any "special" 
or significant math projects for Grades 1, 2, 3, or 4t (e.g. the 
Madison Project, another TV math course, etc.) 



12a. Were the Patterns in Arithmetic telecasts used last year in Grade 4? 
Yes 

No 

b. If yes, did each teacher have a PIA Teacher's Manual? 
Grade 4 Y es 

No 

c. If yes, did each child have a PIA Pupil Exercise Book? 
Grade 4 Yes 

No 

If not, were the exercises duplicated from one copy, or written on 
the chalkboard? Yes No 



13. Did all the PIA instructional materials (manuals, pupil exercise 

books) arrive in time for the start of the field tep.t on September 28? 

Yes No, specify 



14. Is the TV reception consistently good? 

Yes No, specify 



15. Is there a TV in each PIA classroom? 

Yes No, specify 



16. Have you needed to buy any manipulative materials or audiovisual 
aids as a result of beginning PIA? 



No Yes, "pacify 



18. Are all math materials kept in a central location, such as a "math 
resource center?" No Yes 



If yes, describe 

o 
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What do you, as principal, think are the valuable aspects o£ utlliaing 
the Patterns in Arithmetic series? 



20* What are the most significant problems you have thus far identified in 
utilization of Patterns in Arithmetic for Grades 3 and 4t 



21. Rate as accurately as possible the attitude of your Grade 3 and 4 
teachers toward presenting the Patterns in Arithmetic series: 

very dissatisfied somewhat dissatisfied 

with the series 

somewhat satisfied very satisfied 



Further coiranents: 



PRINCIPAL QUESTIONNAIRE 
Control Schools 



T ^ ncipal 
bvnool 



Date 
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What is the time allotment for mathematics per day (or week) in minutest 
Grade A minutes per 



Who determines the allotment? (state, city. Archdiocese, local school 
board) 

How are the mathematics classes grouped? (Check one per grade.) 

Grade 4 

heterogeneously ^.^ ,. . 

homogeneously by general achievement 

homogeneously by math achievement _______ 

homogeneously by reading level ^ 

If teachers for mathematics are not the homeroom teachers, how were they 
selected? (e.g., because they expressed interest, have special talents 
or training, etc.) 



Is there any auxilliary aide or mothers' assistance available to these 
teachers? Yes No 

If so, how many hours per week? - — 

In the last two years, has your school been involved in any "special" or 
significant math projects? (e.g., the Madison Project, another TV math 
course, etc.) For what grade levels? 



Were the Patterns in Arithmetic telecasts used last year in your school? 

Yes No 



For what grade (s)? 



If yes » were the PIA exercise books used? Yes 
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B. HavG your teac'nors occess to (and have they used) any of the PIA 
telecasts this^ year: 

Grade 4 Yes No 

8ii» Are the PIA exercise books being used? €rade 4 



CoTnments: 



9« Are there i&ath manipulative materials in a central location ^ such as a 

^^learning center,'* in your school? ^^^^^.^^^^.^^^ No 

9a» If no, are there such materials in each classroom? , Yes No 

10. Please estimate the percentage of children in this school who fall into 
each of these groups: 

Black (Negro) White (except Puerto Rican, Cuban, 

etcO 

Puerto Mean, Cuban, . Other (specify) . 

etc« 



10a* About what percent of the Puerto Rican, Cuban, etc^, children speak 
primarily Spanish at home? 0-25% 26-50% 

(Indicate the percent when 

possible.) 51-75% 76-100% 

11« Can you estimate what percent of the parents of children in this school 
have a median annual income of below $4000 or are on welfare? (Indicate 
the ^^ercent when possible % 

0-25% 26-50% 51-75% 76-100% 

"■^HHMM te^MMHto talMBtaWto MIBIHBK 

12, Can you estimate what percent of the children in this school are one 
year or more belov grade level according to standardized tests? 
(Indicate the percent when possible.) 

in reading? 0-25% in math? 0-25% 

26-50% 26-50% 

51-75% 51-75% 

76-100% 76-100% 



ERIC 



103 



13. Indicate » in general, the attitude of your teachers toward participation 
as a control school in this study. 

willing, cooperative, positive ____ neutral, indifferent 
negative, uncooperative, antagonistic 

Comment i ._. 



14. Do you feel your teachers are working harder or spending more time on 
mathematics this year because of their participation in this study? 

Yes No 

Comments 



General Comments; 



Observation Guide 
PI A Schools 




School 

Teacher 

Grade 



Observer 



Length of Visit 



to 



DIRECTIONS: Please complete as fully as possible « noting any unusual disturbances^ 
discipline problems, or technical disruptions which may affect what ycu observe* 



la* Was there a Pretelecast Activity directly related to the telecast? 
Yes ^^^^^^^^^^^ No 

b» If yes^ was the activity suggested in the PIA manual? 

Yes No 

2« Did the class appear attentive and interested in the telecast? 

Yes, all students Yes, most Yes, some No, very few 

3* Rate the technical quality of the telecast (picture st^le, sound 
clear, etc) 



Good 



Fair 



Poor 



4a* Was there a Post telecast Activity directly related to the telecast? 

Yes No 

b* If yes, was the activity suggested la the PIA manual? 
Yes No 
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5, What happened itnoediately after the Posttelecast Activity? (Check 
as many as apply*) 

for all students for some students 

Page« in the PIA exercise 

book were assigned 

Pages in a supplementary 

textbook were assigned m..^ 

Teacher-made worksheets 

were assigned ^ 

Follow-up activities from 
the PIA manual were assigned 



Comments : 



6* How many TV sets were used for the te3ecast? ^^^^^^^^^^^^^^^^^^^ 
7» How many students were in the math class? 

8% What happened in general? (Check as maay as apply 

for all students for some students 

Kie teacher retaught or reviewed ..^^^ 

the objectives of the lesson 

Follow-up Activities from the PIA 
manual were done 



Te^^wh^r-made worksheets were assigned 

Exercises from the FIA exercise book 
were assigned 

Other 



9. Was there any evidence of individualization in this classroom? (e,g* did 
children do different exercise pages, or a differing amount of problems 
on the same exercise page, or work in groups with different goals, etc.) 

Yes No 



Comment: 
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10» Were any ^*cues^' from the TV Teacher follo^^e»l through on oy the classroom 
teacher (e*gt making a reference to .]uestion.^> posed by the TV Teacher)? 

Yes No 



11« What manipulative and other audio^visual materials vere obviously 

available in the room? Which were actually used while you were there? 

Obviously available Used 

flashcards ........^^ 

number line ^ 
count ers 

graph paper 

fraction pie ^^.^...^^^.^ 
geometric shapes 

or 3-D) 

rulers . 
place value chart ^^...^^ 
play money 

flannelboard ^^^^^.^^^^ 
other: 



12b Was there evidence of special emphasis on mathematics in this classroom? 
(e.g- a math materials table, a math bulletin boards a math "art" display, 
etc.) 

Yes No 

... Comment : 



erJc 



13* Is the telecast schedule posted in a location such that children can 
read it? 

Yes No 

14^ What was the overall attitude of the students during the observation 
period? 

openly bored ^ frustrated indifferent, unresponsive 

enthusiastic, interested other (specify) t 



15 « What was the overall attitude of the teacher during the observation 
period? 

very enthusaistlc somewhat enthusiastic 



somewhat unenthusiastic very unenthusiastic 

Comments (observer's general inmresslons) ; use back: 
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Observation Guide 
Control Schools 



School 



Teacher* 
Grade 



Observer 



*Note if mathematics teacher 
rather than homeroom teacher 

Length of Visit to 



DIRECTIONS: Please complete as fully as possible, noting any unusual distur- 
bances or discipline problems which may affect what you observe. 



1. How m^y students were in the mathematics class? . — 

2. Did an aide assist with the class? Yes No 

2a. For what portion of the class? . — 

3. What happened in general? (Check as many as apply.) 



For all students For small groups 
in the class of students 



The teacher introduced or reviewed 
a concept (s) or skill (s). 

Exercises from the text (or workbook) 
were assigned. 

Worksheets or chalkboard exercises 
were assigned. 

Activities with manipulative objects 
were used. 



Other 



Comment : 
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A» Wa^ there evidence of individualization in this classroom? (eig»i did 
children do different exercise pages ^ or a differing amount of problems 
on the same exercise page^ or work in groups on different activities > etc*) 

Yes No 



Specify: 



5» What manipulative and other audio^visual materials were obviously avails- 
able in the room? Which were actually used while you were there? 

Obviously Available Used 

flashcards 

number line 

counters ^^.^-...^ ZZZZ 

fraction pie ^^^^ 

geometric shapes (2-D or 3-D) 
rulers 

place value chart 

play money ^^^^ 
flannelboard ^^^^ _ ^ 

other 



6. Was there evidence of special emphasis on mathematics in this classroom? 
(e.g,^ a math materials table^ a math bulletin board, a math **art" display , 
etc«) 

Yes No 

Specify 5 



7. What was the overall attitude of the students during the observation period? 

openly bored > frustrated indifferent, unresponsive 

^ enthusiastic, interested other (specify): ^^^^^^^^.^^^^^ 

8« What was the overall attitude of the teacher during the observation period? 
very enthusiastic ^^^—^ somewhat enthusiastic 

sotaewhat unenthusiastic ^ very unenthuslastic 

Comments; (observer's general impressions) use back if necessary » 



Hcbn Bain 

Past Prtftident 

Kmtional Sducation Association 

Imtitul^ for the Study of Intellectual Behavior 
University of Colorado 

Sue Buel 

^ DiiBemination and InataUation Service* 

Northwest Re^onal Eduoational Laboratory 

Francis S» Chase 
m Professor Emeritus 

University of Chicago 

George £« Dickson 

College of Education 
University of Toledo 



Chester W« Harris 

Graduate School of Education 
University of California 

Hugh J. Scott 
Consultant 

National Evaluation Committee 

H. Craig Sipe 

Department of Instruction 
State University of New York 

C. Wesley Sowards 

Dean of Education 

Florida International University 

Joanna Williams 

Professor of Psychology and Education 
Columbia Univerti^ 



Executive Committee 



William Bush 

Director^ Program Planning and Management 

Deputy Director, RAD Center 

M. Vere DeVault 
Professor 

School of Education 

Herbert J. Klausmeier 
Principsl Investogator 
R&D Center 

Joel R. Levin 

Principal Investigator 
RAD Center 



Donald N. Mclsaac 

Associate Dean^ School of Education 
University of Wisconsin 

Richard A. Rossmiller, Committee Chairman 
Director 
RAD Center 

Len VanEss 

Associate Vice Chancellor 
University of Wisconsin-Madison 

Dan Woolpert 

Director, Management Systems 
RAD Center 



Feciiliy of Principal Invesligslovs 

Vernon Allen 
Professor 
Psychology 

B. Dean Bowles 

Associate Professor 
Educational Administration 

Frank H. Farley 

Associate Professor 
Educational Psychology 

* Marvin J. Fruth 

^ Associate Professor 

Educational Administration 

« John G. Harvey 

Associate Professor 
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Frank H. Hooper 

Associate Professor 
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Herbert J. Klausraeier 
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Gisela Labouvie 

Assistant Professor 
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Joel R. Levin 

Associate Professor 
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Professor 
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Professor 
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Professor 
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Assistant Professor 
Educational Administration 

Richard L, Venesky 
Associate Professor 

Computer Science 
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